The present invention relates to a musical score recognition method and apparatus for processing an image taken from a given musical score text by means of an image scanner or the like so as to recognize staffs, notes, symbols and their positions in the score, to thereby produce performance information containing a tone pitch, a sounding timing and a sounding duration, according to the recognition results.
Recently, it has been attempted to construct a system composed in combination of the musical score recognition apparatus and a sound source such as an electronic musical instrument provided with a Musical Instrument Digital Interface (MIDI). The system operates such that the image data taken from the given musical score text by the image scanner is processed to recognize and extract score information to produce an MIDI data according to the recognized score information, and such that the MIDI data is fed to the MIDI sound source to thereby achieve automatic performance according to the given musical score text by sequential data processing. In such a system, the typical data processing includes the following steps of: (1) sampling and recording a musical score image data; (2) detecting and recognizing staffs; (3) provisionally erasing the staffs; (4) detecting and recognizing bars; (5) provisionally erasing the bars; (6) detecting notes; (7) recognizing the notes; (8) erasing the notes; (9) detecting symbols; (10) recognizing the symbols; (11) erasing the symbols; (12) forming performance data; and (3) producing MIDI data to effect automatic performance.
The step (2) of detecting and recognizing the staffs is carried out such that the recorded image of the musical score is projected into a horizontal direction to calculate a histogram of mark dots piled on a vertical axis. Then, horizontally extending line components arranged at an equivalent interval are detected according to periodical peaks of the histogram to thereby recognize the staffs. The vertical positions or levels of the recognized staff lines are utilized as a basis for determine a pitch of notes recognized by the step (6). The detected staff lines are temporarily erased at the step (3) to avoid ill affect to the later recognition process of the notes and symbols. Further, the detected bars in the step (4) are utilized to recognize a plurality of parts which are concurrently performed. The detection and recognition of the notes and symbols can be conducted by a known pattern matching method. The step (11) of erasing the symbols may be eliminated since the MIDI data can be formed according to the recognition results up to the step (10). However, the symbol may be preferably erased for facilitating the data processing.
The above noted conventional score recognition process is designed for a high performance computer to execute complicated digital image processing. Thus, a personal computer could not execute such a complicated digital image processing at a practical speed of operation. In order to realize practical use of the personal computer in the musical score recognition and subsequent automatic performance, previously various attempts have been conducted to simplify data processing schemes. However, the prior art has still the following drawbacks:
First, in case that a simplified pattern recognition method is adopted for discrimination of notes and symbols in view of reduction in a computation amount, inclination of the musical score image generated at sampling operation would hinder accurate determination of the staff line levels and accurate recognition of the notes and chords.
Second, it would be necessary to provisionally erase the staff lines for applying the conventional simplified pattern recognition method to the discrimination of notes and symbols, because the presence of the staff lines would hinder the effective application of the simplified method. However, excessively thorough erase of the detected staff lines might adversely mutilate objects on the staff lines to thereby hinder correct recognition of the notes and symbols.
Third, the simplified pattern recognition method might fail to discriminate between note heads and note beams, because of general similarity therebetween.
Fourth, the use of the conventional simplified pattern recognition method might suffer from serious pattern recognition error in case that the original score image contains distortions, blurs and other noises.
Fifth, the musical score normally contains a chord composed of multiple notes which are concurrently sounded. Further, the multiple notes may have a different time length or duration. Moreover, a rest may be interposed between adjacent notes. Thus, it would be difficult to correctly determine actual note-on and note-off timings according to the recognition results of the notes, rests and other symbols.